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Introduction 


Were breaking tradition! We were so inspired Whether you need to differentiate FMs from 

by the Al energy in the market today that we LLMs or fine-tuning from few-shot learning, our 
wanted to bring you this Special Edition, glossary helps navigate commonly used terms 
“Ultimate Generative Al Glossary,” instead of to help accelerate your generative Al readiness. 
waiting until year-end. Filled with terms you Enjoy, and we'll see you again in about six 

need to know, this glossary will help months! 


developers, data scientists, and domain experts 
understand and make sense of the whirlwind of 
jargon that has invaded our daily lives. 
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A. 


Adversarial Autoencoder (AAE) - 
Neural networks and models 


A type of autoencoder which combines the 
principles of adversarial loss, integral to GANs, and 
the architecture of an autoencoder. This 
combination empowers the model to learn complex 
distributions of data effectively. 


Adversarial Machine Learning - 
General concept 


This machine learning strategy strengthens models 
by training them with adversarial inputs. It is vital for 
spam filtering, virus detection, and biometric 
identification applications. By challenging models 
with adversarial data, these models become more 
resilient and, thus, more practical in real-world 
situations. 


Agents - Nodes and Prompts 


In the context of Al, agents are software that can 
independently perform specific tasks without 
human intervention. They often employ various 
tools, like calculators or web browsing, to process 
data and develop solutions. 


AGI (Artificial General Intelligence) - 
Into the future of Al 


As defined by some Microsoft researchers, AGI 
represents an artificial intelligence that matches 
human intellectual capabilities across a wide range 
of tasks. Despite its universally acceptable definition 
being debatable, AGI is a significant frontier in Al 
research. 
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ASI (Artificial Super Intelligence) - 
Into the future of Al 


A theoretical future phase of artificial intelligence 
surpassing human intelligence across all aspects. 
ASI refers to Al that outperforms the most proficient 
human brains in practically all economically valuable 
work, including scientific research, general wisdom, 
and social skills. 


Audio Synthesis - Cross-Domain Al models 


This involves using Al to create new, artificial sounds 
or voice outputs. Such sounds can be as simple as a 
specific tone or as complex as a mimicked form of 
speech. 


Auto-regressive Models - Unsupervised 
learning and anomalies 


These are generative models that produce data by 
conditioning each element's probability on previous 
elements in a sequence. For example, WaveNet and 
PixelCNN are autoregressive models for creating 
music and images, respectively. 


Autoencoder - Neural networks and models 


An autoencoder is an artificial neural network 
utilized for learning efficient encodings of input 
data. It has two crucial components: an encoder 
that compresses the input data and a decoder that 
reconstructs the data from its reduced form. 


Autoregressive Generative Models - 
Unsupervised learning and anomalies 


These models predict the distribution of subsequent 
sequence elements using prior sequence elements 
to implicitly establish a distribution across 
sequences using Conditional Probability's Chain 
Rule. The main architectures for autoregressive 
models are causal convolutional networks and 
recurrent neural networks. 


B. 


BERT (Bidirectional Encoder 
Representations from Transformers) 
- Neural networks and models 


BERT, developed by Google, is a pre-trained 
transformer-based language model. It stands out for 
its bidirectional training approach, which allows it to 
understand the context of a word based on all of its 
surroundings (left and right of the word). 


Bloom - Neural networks and models 


Developed by The BLOOM project, Bloom is a large- 
scale language model that can execute a vast array 
of natural language understanding and generation 
tasks accurately. 


G 


Chatbot - Natural Language processing 


A computer program designed to simulate human 
conversation through either text or voice 
interactions. These tools often use natural language 
processing techniques to understand user input and 
provide relevant responses. 


ChatGPT - Natural Language processing 


Developed by OpenAl, ChatGPT is a specialized 
large-scale language model that generates human- 
like text. It's a popular choice for developing Al- 
powered chatbots due to its convincing 
conversation-generation capabilities. 


CLIP (Contrastive Language—Image 
Pretraining) - Cross-Domain Al models 

This involves using Al to create new, artificial sounds 
or voice outputs. Such sounds can be as simple as a 


specific tone or as complex as a mimicked form of 
speech. 
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Conditional GANs (cGANSs) - Generative 


adversarial networks 


These are a type of GAN where a conditional 
variable is introduced to the input layer, allowing the 
model to generate data conditioned on certain 
factors. This augmentation provides the model the 
Capability to generate data with desired 
characteristics. 


Crossmodal - Cross-Domain Al models 


Cross-modal learning, on the other hand, refers to 
using information from one modality to understand 
or make predictions in another modality. This could 
involve translating or transforming the data in some 
way. For example, a cross-modal learning system 
might be designed to accept text input and output a 
related image or vice versa. 


CycleGAN - Generative adversarial networks 


A type of GAN that can translate an image from a 
source domain to a target domain without paired 
examples. It's particularly useful in tasks like photo 
enhancement, image colorization, and style transfer 
for unpaired photo-to-photo translation. 


CycleGAN 


D. 


DALL-E 2 - Image processing and generation 


This is an updated version of DALL-E, an Al model 
developed by OpenAl to generate images from 
textual descriptions. It's an excellent example of a 
multi-modal Al system. 


Data Distribution - Learning strategies 


In machine learning, data distribution refers to the 
overall layout or spread of data points within a 
dataset. In the case of generative models such as 
GANs, the generator seeks to mimic the actual data 
distribution. 


Deepfake - Image processing and generation 


Synthetic media in which a person in an existing 
image or video is replaced with someone else's 
likeness using machine learning techniques. While 
they could serve interactive entertainment 
purposes, deepfakes may mislead viewers, often 
with unintended consequences. 


Diffusion - Cross-Domain Al models 


In Al, ‘diffusion’ refers to a technique used for 
generating new data by starting with a portion of 
actual data, then gradually adding random noise. 
This process is generally reversed, with a neural 
network trained to predict the reverse process of 
noise addition to the data. 


Discriminator - Generative adversarial 
networks 


In a GAN, the discriminator is the component that 
tries to differentiate real data instances from the 
fictitious ones fabricated by the generator. It helps 
refine the generator's ability to create realistic data. 
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Emergence/Emergent Behavior - 
General concept 


("sharp left turns," intelligence explosions). In 
artificial intelligence, emergence refers to complex 
phenomena that arise from simple rules or 
processes. Radical concepts like "sharp left turns" 
and "intelligence explosions" denote sudden, 
dramatic developments in Al, often related to AGI's 
emergence. 


Embedding- Embeddings and latent spaces 


An embedding represents data in a new form, often 
a vector space, facilitating comparisons and 
calculations with other data points. Similar items 
should have similar embeddings, making it an 
essential feature for many Al tasks, like 
recommendation systems and natural language 
processing. 


F. 


Foundation Model - General concept 


In Al, foundation models are large-scale Al models 
trained on diverse and extensive data meant to be 
fine-tuned or adapted for more specific tasks. 
These are called foundation models, as they offer a 
robust and broad foundation that can be built upon 
for various Al tasks. 


Fine-tuning - Learning strategies 


A form of transfer learning wherein a pre-trained 
model is slightly modified or adjusted to perform a 
new task. This process allows for more efficient use 
of the pre-trained models by adjusting them to solve 
tasks similar to ones they were originally trained on. 


Few-Shot Learning 
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Few-Shot Learning - Learning strategies 


A machine learning method where the model learns 
to perform a task from a few examples per class. For 
instance, it can correctly categorize new data after 
being shown only a few samples from each 
category. 


G. 


Generative Pre-trained Transformer 
(GPT) - Neural networks and models 


GPT is a family of neural network models trained to 
generate content. These models are pre-trained on 
vast amounts of text data, allowing them to 
generate coherent and relevant text based on user 
prompts. GPT models can automate content 
creation and analyze customer feedback for 
insights, fostering personalized interactions. 


GPT-1, GPT-2, GPT-3, GPT-4:** GPT-1 
GPT-2, GPT-3, and GPT-4 - Neural 


networks and models 


are progressive versions of the generative pre- 
trained transformers developed by OpenAl. Each 
model sees improvements and expansions on its 
predecessors, offering advanced text generation 
Capabilities and greater application versatility. 
GPT-3, for instance, is an extremely sophisticated 
model known for its wide-ranging applicability, 
including translation, question-answering, and text 
completion tasks. 
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GPT-J - Neural networks and models 


GPT-J is an open-source large language model 
developed by EleutherAl in 2021. It is a generative 
pre-trained transformer model with 6 billion 
parameters, similar to GPT-3, but with some 
architectural differences. GPT-J was trained on a 
large-scale dataset called The Pile, a mixture of 
sources from different domains. 


GPT-Neo - Neural networks and models 


GPT-Neo is a family of transformer-based language 
models from EleutherAl based on the GPT 
architecture. It is an open-source alternative to 
GPT-3 that can generate natural language texts 
using deep learning. GPT-Neo has two pre-trained 
models with 1.3 billion and 2.7 billion parameters 
and was also trained using The Pile dataset. 


Generative Adversarial Networks 
(GANS) - Generative adversarial networks 


Introduced by lan Goodfellow and his colleagues in 
2014, GANs are generative models involving two 
sub-models: a generator and a discriminator. The 
generator creates new data instances, and the 
discriminator evaluates them for authenticity. The 
systems function adversarially, resulting in the 
generator producing progressively better data. 


Generator - Generative adversarial networks 


In a Generative Adversarial Network, the generator 
is the component that creates new instances of 
data by learning to mimic the real data distribution. 


Generative Models for Images - 
Image processing and generation 


These are generative models like GANs, VAEs, and 
DALL-E, trained on image data and capable of 
generating new images that reflect the patterns 
found in the training data. 


H. 


Hallucination -Unsupervised learning and 
anomalies 


In Al, a hallucination occurs when a model makes 
erroneous conclusions and generates content that 
doesn't correspond to reality. These erroneous 
outputs indicate problems in the workings of the Al 
model. Team vigilance is necessary to maintain the 
accuracy and reliability of Al systems in identifying 
and mitigating hallucinations. 


Image Translation - Image processing 
and generation 


A task in computer vision where the goal is to map 
or translate one image into another, often using a 
model known as GANS. For example, translating a 
daytime scene into a nighttime scene. 


Inpainting - Image processing and generation 


A generative task where the Al is meant to fill in 
missing or corrupted parts of an image. Typical 
applications include photo restoration and the 
completion of unfinished art. 


L. 


Langchain - Nodes and Prompts 


Langchain is a concept in Al and machine learning 
that affects reasoning capability. When prompting 
an LLM, the "chain-of-thought" technique improves 
the model's reasoning by breaking tasks into 
smaller, discrete steps. A more complex approach, 
"tree-of-thought," allows logical steps to branch and 
backtrack. 
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Large Language Models (LLMs) - 


Natural Language processing 


Large-scale Al models trained on extensive text 
data, such as GPT-3 and BERT. They can respond to 
prompts, generate text, answer questions, create 
poetry, and even generate code. This ability can 
enable personalized and authentic customer 
interactions and assist in automating customer- 
facing content. 


Latent Space- Embeddings and latent 
spaces 


In generative models, latent space refers to a 
compressed input data representation. It is the 
transition medium between the noise injected into 
the GAN' generator and its output. 


M. 


Machine Learning Bias - 
General concept 


Bias in machine learning can occur from intentional 
or unintentional biased data or algorithms making 
incorrect assumptions, leading to skewed decisions. 
Understanding and addressing this bias ensures fair 
and accurate treatment for all customers. 


MidJourney - Image processing and 
generation 


This dataset and model, developed by OpenAl, 
specializes in rendering images of outdoor scenes 
mid-journey, showcasing the creative ability of Al in 
the generation of visual content. 


Mixture of Experts - Learning strategies 


Is a machine learning method where specialized 
models, or “experts”, handle different parts of data 
distribution. The final prediction is a blend of these 
expert outputs, adjusted by a “gating” system that 
determines each expert’s relevance. This leverages 
individual strengths to form a more robust model. 


Mode Collapse - Generative adversarial 
networks 


This phrase refers to a situation when the Generator 
in a Generative Adversarial Network begins to 
produce the same output (or a narrow set of 
outputs) repetitively rather than generating diverse 
outputs. It destabilizes the learning process and 
poses a Challenge in GAN training. 


Multi-modal Al - General concept 


This type of Al has the capability to process and 
understand inputs from different data types, like 
text, speech, images, and videos. Thus, these Al 
models can deal with diverse data inputs, enhancing 
their applicability in various contexts. 


Multimodal - Cross-Domain Al models 


A multi-modal learning model makes predictions by 
accepting and analyzing various types of input, such 
as audio and video data, improving its 
understanding of scenarios like movie scenes. 
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N. 


NeRF (Neural Radiance Fields) - Image 
processing and generation 


A method for creating a three-dimensional scene 
from two-dimensional images using a neural 
network. NeRF can create a photorealistic 
rendering, synthesize views, and offer more 
Capabilities in understanding and reconstructing 
scenes from 2D images. 


O. 


Outpainting - Image processing and 
generation 


A generative task where the Al is asked to extend 
the existing content of an image. It fills the areas 
beyond the image boundaries with plausible content 
that seamlessly connects with the original image 
context. 


P. 


Plugins / tools - Into the future of Al 


Al agents such as LLMs may have the ability to use 
‘tools’ via APIs that give them new capabilities. For 
example, LLMs equipped with web search 
Capabilities can access data not present in their 
training dataset, which can significantly reduce the 
risk of hallucinations. 


PPO (Proximal Policy Optimization) - 


Unsupervised learning and anomalies 


A reinforcement learning algorithm developed by 
OpenAl that is designed to achieve high 
performance while reducing the amount of 
computation required for each update. 


Prompt - Nodes and Prompts 


A prompt is the initial input or direction given to an 
Al model to execute a task or answer a query. It sets 
the starting context for the model's generation 
process. 


Prompt Engineering - General concept 


The practice of carefully crafting queries to an Al 
model to get the best possible output. Strong and 
relevant prompts are essential for enabling 
generative Al tools to deliver valuable outputs, 
enhancing both the customer and team experience. 


Prompt Tokens d Asal Torens / 
Completion - Nodes and Prompts 


These terms relate to how the Al uses tokens or 
units of data as input or output. A prompt token 
starts the model's data generation process, the 
model chooses sampled tokens during this process, 
and completion signifies the model's output 
following the prompt. 


R. 


Reinforcement Learning - General 
concept 


In this kind of Al training, an agent learns to make 
decisions by interacting with an environment and 
receiving feedback in the form of rewards or 
punishments. The objective is to maximize the total 
reward earned and learn optimal actions in different 
situations. 


-e Reinforcement Learning 


Retrieval Augmented Generation 


(RAG) 


RAG is a specific term that describes supplementing 
a prompt with additional information based on web 
searches or internal/private document queries. It 
can help mitigate ‘hallucination’ issues where an Al 
model generates content without sufficient 
grounding in reality. 


- Cross-Domain Al models 


RLHF (Reinforcement Learning from 
Human Feedback - General concept 


This technique incorporates human feedback into 
the learning process of an Al model. Evaluators 
provide feedback on the model's outputs, which can 
help improve the model's performance over time. 


S. 


Self-Supervised Learning - Unsupervised 
learning and anomalies 


This is a type of machine learning that trains an 
algorithm to analyze and infer from test data that 
hasn't been classified, labeled or categorized, as 
opposed to supervised learning that usually involves 
training data which has been labeled, classified and 
categorized. 


Sequence Generation - Natural Language 
processing 


This is a task in natural language processing where 
the model generates a sequence of words or 
symbols, such as in text generation. It's one of the 
capabilities of auto-regressive language models, 
including GPT and BERT. 


Style Transfer - Image processing and 
generation 


This generative model application involves capturing 
the artistic style of one image (the style source) and 
transferring it onto another image (the content 
source). 


StyleGAN - Generative adversarial networks 


Developed by Nvidia, StyleGAN is a GAN-based 
model known for its high-quality and consistent 
outputs. Particularly, it gained attention for its 
capability to generate hyper-realistic images of 
human faces. 


Super Resolution - Image processing and 
generation 


An application of generative models that involves 
increasing the resolution of an image. Using these 
models, lower-quality images can be enhanced 
successfully. 


Symbolic Artificial Intelligence - 
General concept 


A type of Al that leverages symbolic reasoning 
mechanisms to solve problems and represent 
knowledge. It typically involves the usage of 
symbols to represent concepts and the relationships 
between them. 


System Prompt - Nodes and Prompts 


This refers to the predefined instructions that set 
the general behavior of an Al system, like a chatbot. 
It begins every interaction and influences how the Al 
responds to user inputs. 
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T. 


Token - Natural Language processing 


In the context of neural networks, tokenization is the 
process of encoding text into numerical values. 
Tokens may represent letters, groups of letters, or 
whole words. 


Transformer - Neural networks and models 


Transformers are a model architecture used in 
machine learning, mainly for handling sequential 
data in natural language processing tasks. They 
leverage attention mechanisms to weigh the 
influence of different input parts differently. This 
varies with the traditional RNN or LSTM models as it 
allows for better parallelization and handles long- 
range dependencies more effectively. 


U. 


Unsupervised Learning - Unsupervised 
learning and anomalies 


This is a type of machine learning that trains an 
algorithm to analyze and infer from test data that 
hasn't been classified, labeled or categorized, as 
opposed to supervised learning that usually involves 
training data which has been labeled, classified and 
categorized. 


W. 


Weakly Supervised Learning - 
Unsupervised learning and anomalies 


This is an approach to supervised learning in which 
the training data is noisy, limited, or, imprecise; 
however, these weakly labeled samples are often 
easier and cheaper to obtain, resulting in larger 
effective training sets. 
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V. Z. 


Zero-Shot Learning - Learning strategies 


A type of machine learning where the model can 
Unlike traditional autoencoders, which can learn any Make predictions about data it has never 

function to reconstruct data, a VAE places additional encountered during its training. It leverages 
constraints on encoded representations, so they similarities between what it has seen and the novel 
learn parameters of a probability distribution data to make predictions. 


representing the data. 


Variational Autoencoder (VAE) - 
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Images Generated 
with Midjourney 
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